{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2 style='color:purple' align='center'>Build our first neural network in Keras for image classification problem</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We will use keras fashion MNIST dataset. This consist of 60000 28X28 pixel images and 10000 test images, these images are classified in one of the 10 categories shown below**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"slide1.png\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Each image is 28 x 28 pixel in dimension**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='slide2.png' />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'tensorflow'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "keras.backend.backend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "fm = keras.datasets.fashion_mnist\n",
    "(X_train, y_train), (X_test, y_test) = fm.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 28, 28)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   1,\n",
       "          0,   0,  13,  73,   0,   0,   1,   4,   0,   0,   0,   0,   1,\n",
       "          1,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   3,\n",
       "          0,  36, 136, 127,  62,  54,   0,   0,   0,   1,   3,   4,   0,\n",
       "          0,   3],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   6,\n",
       "          0, 102, 204, 176, 134, 144, 123,  23,   0,   0,   0,   0,  12,\n",
       "         10,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0, 155, 236, 207, 178, 107, 156, 161, 109,  64,  23,  77, 130,\n",
       "         72,  15],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   1,   0,\n",
       "         69, 207, 223, 218, 216, 216, 163, 127, 121, 122, 146, 141,  88,\n",
       "        172,  66],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   1,   1,   1,   0,\n",
       "        200, 232, 232, 233, 229, 223, 223, 215, 213, 164, 127, 123, 196,\n",
       "        229,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "        183, 225, 216, 223, 228, 235, 227, 224, 222, 224, 221, 223, 245,\n",
       "        173,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "        193, 228, 218, 213, 198, 180, 212, 210, 211, 213, 223, 220, 243,\n",
       "        202,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   1,   3,   0,  12,\n",
       "        219, 220, 212, 218, 192, 169, 227, 208, 218, 224, 212, 226, 197,\n",
       "        209,  52],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   6,   0,  99,\n",
       "        244, 222, 220, 218, 203, 198, 221, 215, 213, 222, 220, 245, 119,\n",
       "        167,  56],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   4,   0,   0,  55,\n",
       "        236, 228, 230, 228, 240, 232, 213, 218, 223, 234, 217, 217, 209,\n",
       "         92,   0],\n",
       "       [  0,   0,   1,   4,   6,   7,   2,   0,   0,   0,   0,   0, 237,\n",
       "        226, 217, 223, 222, 219, 222, 221, 216, 223, 229, 215, 218, 255,\n",
       "         77,   0],\n",
       "       [  0,   3,   0,   0,   0,   0,   0,   0,   0,  62, 145, 204, 228,\n",
       "        207, 213, 221, 218, 208, 211, 218, 224, 223, 219, 215, 224, 244,\n",
       "        159,   0],\n",
       "       [  0,   0,   0,   0,  18,  44,  82, 107, 189, 228, 220, 222, 217,\n",
       "        226, 200, 205, 211, 230, 224, 234, 176, 188, 250, 248, 233, 238,\n",
       "        215,   0],\n",
       "       [  0,  57, 187, 208, 224, 221, 224, 208, 204, 214, 208, 209, 200,\n",
       "        159, 245, 193, 206, 223, 255, 255, 221, 234, 221, 211, 220, 232,\n",
       "        246,   0],\n",
       "       [  3, 202, 228, 224, 221, 211, 211, 214, 205, 205, 205, 220, 240,\n",
       "         80, 150, 255, 229, 221, 188, 154, 191, 210, 204, 209, 222, 228,\n",
       "        225,   0],\n",
       "       [ 98, 233, 198, 210, 222, 229, 229, 234, 249, 220, 194, 215, 217,\n",
       "        241,  65,  73, 106, 117, 168, 219, 221, 215, 217, 223, 223, 224,\n",
       "        229,  29],\n",
       "       [ 75, 204, 212, 204, 193, 205, 211, 225, 216, 185, 197, 206, 198,\n",
       "        213, 240, 195, 227, 245, 239, 223, 218, 212, 209, 222, 220, 221,\n",
       "        230,  67],\n",
       "       [ 48, 203, 183, 194, 213, 197, 185, 190, 194, 192, 202, 214, 219,\n",
       "        221, 220, 236, 225, 216, 199, 206, 186, 181, 177, 172, 181, 205,\n",
       "        206, 115],\n",
       "       [  0, 122, 219, 193, 179, 171, 183, 196, 204, 210, 213, 207, 211,\n",
       "        210, 200, 196, 194, 191, 195, 191, 198, 192, 176, 156, 167, 177,\n",
       "        210,  92],\n",
       "       [  0,   0,  74, 189, 212, 191, 175, 172, 175, 181, 185, 188, 189,\n",
       "        188, 193, 198, 204, 209, 210, 210, 211, 188, 188, 194, 192, 216,\n",
       "        170,   0],\n",
       "       [  2,   0,   0,   0,  66, 200, 222, 237, 239, 242, 246, 243, 244,\n",
       "        221, 220, 193, 191, 179, 182, 182, 181, 176, 166, 168,  99,  58,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,  40,  61,  44,  72,  41,  35,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0]], dtype=uint8)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1c50bc242b0>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAECCAYAAAAYUakXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFGZJREFUeJzt3X9w3HWdx/HXO5tN0h9J27QllFopVVBQjqKRIjCK4g9k\nbg5Qj5FxPJxhrsydx6nnHzqcN3J/3A3jCB5zjt4V4cQZ4UZHOJFhVCgi4g+kFI6W9rD8KJTSpi2F\npj+SZjd53x9ZPkbovj9pNt3vVp6PmU6Tfe1mP/lu8sp39/vZz9fcXQAgSW1FDwBA66AQACQUAoCE\nQgCQUAgAEgoBQFJIIZjZ+Wb2hJk9aWZfKmIMETPbbGbrzOxRM1vTAuO5ycx2mNn6CZf1mtndZrap\n9v+8Fhvf1Wa2tbYNHzWzCwoc3xIz+7mZbTCzx83ss7XLW2IbBuNr+ja0Zs9DMLOSpN9L+qCk5yU9\nJOlSd9/Q1IEEzGyzpH5331X0WCTJzN4jaZ+k77r722uXfVXSbne/plaq89z9iy00vqsl7XP3rxUx\nponMbJGkRe6+1sy6JT0s6SJJn1YLbMNgfJeoyduwiD2EMyQ96e5Pu/uIpP+WdGEB4zhquPv9kna/\n6uILJd1c+/hmjf8AFaLO+FqGu29z97W1j/dK2ihpsVpkGwbja7oiCmGxpC0TPn9eBX3zAZd0j5k9\nbGYrix5MHX3uvq328XZJfUUOpo4rzeyx2lOKwp7STGRmSyWdLulBteA2fNX4pCZvQ15UPLRz3H25\npI9I+kxtl7hl+fjzvlabg/4tScskLZe0TdK1xQ5HMrPZkn4o6XPuPjgxa4VteIjxNX0bFlEIWyUt\nmfD5G2qXtQx331r7f4ek2zX+NKfVDNSee77yHHRHweP5I+4+4O6j7j4m6QYVvA3NrKzxX7bvuftt\ntYtbZhseanxFbMMiCuEhSSea2Qlm1iHpE5LuKGAch2Rms2ov7MjMZkn6kKT18a0KcYeky2ofXybp\nRwWO5TVe+UWruVgFbkMzM0k3Stro7tdNiFpiG9YbXxHbsOlHGSSpdvjk3ySVJN3k7v/S9EHUYWbL\nNL5XIEntkm4penxmdqukcyUtkDQg6SuS/kfS9yW9UdKzki5x90Je2KszvnM1vqvrkjZLumLC8/Vm\nj+8cSb+UtE7SWO3iqzT+PL3wbRiM71I1eRsWUggAWhMvKgJIKAQACYUAIKEQACQUAoCk0EJo4WnB\nkhhfo1p5fK08Nqm48RW9h9DSD4oYX6NaeXytPDapoPEVXQgAWkhDE5PM7HxJ12t8xuG33f2a6Pod\n1uldmpU+r+igyuqc8v0faYyvMa08vlYemzT94xvWfo34Qctdb8qFMJWFTnqs11fYeVO6PwBT96Cv\n1qDvzhZCI08ZWOgE+BPTSCEcDQudADgM7Uf6DmqHT1ZKUpdmHum7A9CARvYQJrXQibuvcvd+d+9v\n5RdxADRWCC290AmAwzflpwzuXjWzv5P0U/1hoZPHp21kAJquodcQ3P0uSXdN01gAFIyZigASCgFA\nQiEASCgEAAmFACChEAAkFAKAhEIAkFAIABIKAUBCIQBIKAQACYUAIKEQACRHfAk1tBDLLLrbwJL8\nklSa3xvmL334pDDvueW3Dd1/7vuz9nKYe2WksftvVO7xyWnw8ZPYQwAwAYUAIKEQACQUAoCEQgCQ\nUAgAEgoBQMI8hNcRK5XC3KvVMG9bfkqYb7xidnz7oTBWef8ZYd4+NBbf/mdrwrzheQa5eQ6Z7SuL\n//42Oj5rD36d44c2YQ8BQEIhAEgoBAAJhQAgoRAAJBQCgIRCAJAwD+F1JDxOrfw8hC0fnhvmn3z3\nL8P8VzuXhfmznceGuc8IY7V/4N1hftI3t4Z5dfNz8R1k1hvIbb+c0rx58RVGR+N4cLB+OMmlEhoq\nBDPbLGmvpFFJVXfvb+TrASjWdOwhvM/dd03D1wFQMF5DAJA0Wggu6R4ze9jMVk7HgAAUp9GnDOe4\n+1YzO0bS3Wb2f+5+/8Qr1IpipSR1aWaDdwfgSGpoD8Hdt9b+3yHpdkmvebuau69y93537y+rs5G7\nA3CETbkQzGyWmXW/8rGkD0laP10DA9B8jTxl6JN0u42/R7xd0i3u/pNpGRWOiLHh4YZuP3L6vjD/\n+Jx4PYKutkqY/6ItXu9g671Lwnz0z+LxPXtdd5iPPXJWmM9fH88D6HlkW5jves/iMN/5zniyQF/m\ntBXz7nmqbma7J/erPuVCcPenJZ021dsDaD0cdgSQUAgAEgoBQEIhAEgoBAAJhQAgMZ+Gc8pPVo/1\n+go7r2n397qTOW9A7v38+y45M8w/8uX7wvzkrhfCfO9YV5iPeGMz6b/xxHvDfP/Tc8K8bSSz/TLx\naF98XgWvxH9/562Nv/8ZFw6Eud2wsG722OrrtW/3lsx3wB4CgAkoBAAJhQAgoRAAJBQCgIRCAJBQ\nCAAS5iG0ktw8gkZlHuu3Pxz/ffjovHi9g5xS5uQA+70jzF8endXQ/e+sxushVDLzIL69KV4vYV9u\nnkM1fnw/+L5HwvxjvQ+F+VffdGrd7EFfrUHfzTwEAJNHIQBIKAQACYUAIKEQACQUAoCEQgCQTMfZ\nnzFdmjgn5FA27TsmzF/smR3m26tzw3x+KT5vQnfbUJgvLccnGd85Gs8zKJXj8z6MeCnM//ltPw7z\n4ZPLYV62+LwOZ2XWk/jLDX8V5rP0dJhPBnsIABIKAUBCIQBIKAQACYUAIKEQACQUAoCEeQhIFnbG\n8wS6rBLmHVYN8xcq88J809Bbwvz3g/E8ifP7Hg/zSmaeQW69htw8guPKL4X5sMfzFOKtK53dF88z\neDRz+8nI7iGY2U1mtsPM1k+4rNfM7jazTbX/40cawFFhMk8ZviPp/Fdd9iVJq939REmra58DOMpl\nC8Hd75e0+1UXXyjp5trHN0u6aJrHBaAAU31Rsc/dt9U+3i6pb5rGA6BADR9l8PFVWuu+GmNmK81s\njZmtqehgo3cH4AiaaiEMmNkiSar9v6PeFd19lbv3u3t/WZ1TvDsAzTDVQrhD0mW1jy+T9KPpGQ6A\nImXnIZjZrZLOlbTAzJ6X9BVJ10j6vpldLulZSZccyUG+bmTOy2Cl+Di6V+N5AKV58dHh985dF+Y7\nR3vC/OXRmWE+t3QgzPdWu8J891D89d/auS3M1x5YGuYLO+J5BLnxbx5ZEOYndm4P868OxOcsWdL1\n6tf2/1j1vPfUzfzB34S3fUW2ENz90joRZ1wB/sQwdRlAQiEASCgEAAmFACChEAAkFAKAhPUQWknm\nvAzWHj9cuXkIWy4/OczfPzM+78CvhxeH+cL2vWGeW49gUeeeMO/uGw7z3DyI3vZ4vYe9ozPCfGZb\nPPU+9/2/oyM+r8Tn73lHmHe//cUw7ykHf9/jKS4JewgAEgoBQEIhAEgoBAAJhQAgoRAAJBQCgIR5\nCC3Eyh1hPjYcH4fPWbBuJMx3jcbnDZjbFq8H0JE5b8FIZh7CWb3PhPnOzDyBtUMnhHl3aSjMF7bF\n8wiWlON5AOuGl4T5XfvfHOaX//k9YX7rqg+GecdPfl03M48fu1ewhwAgoRAAJBQCgIRCAJBQCAAS\nCgFAQiEASI6ueQi58xa0x8fRrZTpv7Y4HxvOnIpuLD4On+OVeJ5Ao67/z2+E+Zbq3DDfXonz3HkL\nRjNvyv/t0Jww72qrhPnC9sEwHxyL5zHk7B2LzxuRW+8hN/4vzt8U5rft+UCYTwf2EAAkFAKAhEIA\nkFAIABIKAUBCIQBIKAQASUvNQ2j0vAO54/geHwYu3NCFZ4T5lovieQ6fPP13Yb692h3mjxxYGuZz\nMusJzMqct2DY43kiL4zMC/PccfzceReOycxTGPX47+PWSjy+nNw8jeermfNG/EW8XsPc7x72kF4j\nu4dgZjeZ2Q4zWz/hsqvNbKuZPVr7d0HjQwFQtMk8ZfiOpPMPcfnX3X157d9d0zssAEXIFoK73y9p\ndxPGAqBgjbyoeKWZPVZ7StHYkysALWGqhfAtScskLZe0TdK19a5oZivNbI2Zrako8+YgAIWaUiG4\n+4C7j7r7mKQbJNV9edzdV7l7v7v3l9U51XECaIIpFYKZLZrw6cWS1te7LoCjR3YegpndKulcSQvM\n7HlJX5F0rpktl+SSNku6YjoGk5tn0Kj2RceGeeWEvjDfffLMMD9wbPx+/+UXbAzzT/f9V5jvHO0J\n87LF229LZX6Ynz5zc5jfu+eUMN/VPjvMc/MYzpoVrwfw8li8/Y9rfynMv/jkx8O8b2Z8nP/bx8cH\n0yo+FuZPVOI95D1j8XoKf3/Kz8P8di0M88nIFoK7X3qIi29s+J4BtBymLgNIKAQACYUAIKEQACQU\nAoCEQgCQtNR6CAc/8q4wP+Yfnw7z5T3Ph/kpMx4I8+Gx+P36uffjbxhaHOYHxjrCfNNIPE9iTzU+\nDl+y+Dj4jpF4PYRrn4nX/V99xn+E+ZdfONSbYv+gbYaH+Yuj8TyGj82O1zOQ4sfvijfeH+bLOnaE\n+Z37F4X5C5n1EvrKe8J8aXlnmH+0+/dhPh3zENhDAJBQCAASCgFAQiEASCgEAAmFACChEAAkzZ2H\nYPG5F1b860Phzc/rfjzMD3j8fvPcPIPcceScOe3xuvsHK/Hm3lGJ1zvIOalze5hf3PNomN//jRVh\nfs7wlWH+1Pvj9RxWD8Xv999Zjb//Tzzz/jBf+9ySMD9z6TNhfmr31jDPzQPpLg2HeW69iv1j8c/v\nb4fjeRrTgT0EAAmFACChEAAkFAKAhEIAkFAIABIKAUBi7vF71KfTjGOX+Js+9Q9181Wf+ffw9rfs\nPjPMl3TF56Q9vmNXmM8v7QvznO62+Dj0W8rxceg7978hzO97+a1h/s7uzWFettEwP3fmk2H+6c9/\nIcyrXfF5KQaXxn9/qrPin8We014M8yvffG+Yd2S+/5dH43kGue03txTPQ8nJrWfR3Raf1+LaCy6u\nm/1m83e0Z2hb/ACJPQQAE1AIABIKAUBCIQBIKAQACYUAIKEQACRNXQ+hrSLNHKh/rPXOweXh7ZfN\niNet31WJzzvw032nhvkbZrwU5nNK8XHgN2fWI3h0eG6Y/2Tn28L8uBnxeQkGKnPC/MXKrDA/kHk/\n/o1fvy7Mrx2Iz+twce/aMD+tI55n8PJY/PdrQ+a8FnvHusJ82OP1MvZk5il0Z34+Kh7/upU8nocw\nty2e5zB46vy62ejA5H7Vs3sIZrbEzH5uZhvM7HEz+2zt8l4zu9vMNtX+b2x1EQCFm8xThqqkL7j7\nKZLOlPQZMztF0pckrXb3EyWtrn0O4CiWLQR33+bua2sf75W0UdJiSRdKurl2tZslXXSkBgmgOQ7r\nRUUzWyrpdEkPSupz9221aLukvmkdGYCmm3QhmNlsST+U9Dl3/6NXt3z8HVKHfGeKma00szVmtqZ6\ncH9DgwVwZE2qEMysrPEy+J6731a7eMDMFtXyRZIOeepcd1/l7v3u3t/eGb/KDaBYkznKYJJulLTR\n3Sced7pD0mW1jy+T9KPpHx6AZprMwcmzJX1K0joze2Vh/6skXSPp+2Z2uaRnJV2S+0KlkTF1bzlY\nNx/z+O3a9+6K1wPo69ob5su7t4T5Ewfi49jrho4L87XtbwzzGaVKmM/piNdTmNVef9tJ0oJy/P2f\n0HnInbgkt17AQ8Px9/c3C+8L8+eq8ZHpH+8/Kcw3HIi3/7zMeTHWDca3P1DtCPODo/Gvy3A1nucy\npzN+fN/V+2yYP6FFYb7ztPp/36u/Cm+aZAvB3R+QVO839bzJ3Q2AowFTlwEkFAKAhEIAkFAIABIK\nAUBCIQBImroegvYNqe0Xj9SNf/Czs8Ob/9OFPwjzX2TOW3Dn9vg48eBIvB7Awpnx1OuezDyA3nJ8\n+zmZ4+hdFp/X4aVqPBP0YFv8fv/RukeXx20/GK+38KuxE8O8MlYK84OZPDePY/fIgjA/bsaeMN9b\njddL2Ly3N8x37Zkd5sMz41+3B0bfFObnH/t4mM/YUf/xa4t/dP5wvcldDcDrAYUAIKEQACQUAoCE\nQgCQUAgAEgoBQGLjq581R4/1+gqb+jum93zyzDBf9rdPhPkZc58J87WD8fv9n8sch65kzhtQbovX\n3Z9ZHgnzrsxx+I5SvJ5B26FXuUvGMvMQZpXi8eXWa+hpj9cD6C7FeZvF2y+nlPn+f7dnaUNfvzvz\n/Vc9/vl495ynwvymZ84K8zkXPFk3e9BXa9B3xw+w2EMAMAGFACChEAAkFAKAhEIAkFAIABIKAUDS\n/HkIpQ/Vv8JYfBy9Ufs/tiLMV1z1UJx3x8eJ39oxEOZlxcfRuzLH2We1xYeRhzOPZa79HxhaEuaj\nma9w70snh3klcxx+4EBPmJcz8yxycuf9GKrG60XsGYrXSyi1xdt/+L54vYb5G+J5Jp13xT+fEeYh\nADhsFAKAhEIAkFAIABIKAUBCIQBIKAQASXYegpktkfRdSX2SXNIqd7/ezK6W9NeSdtauepW73xV9\nrUbXQ2h19q74vA9Dx84I884X4/fT7z0+vn3PU/F5H9oOxovzj/3vxjDH0Wuy8xAmc6KWqqQvuPta\nM+uW9LCZ3V3Lvu7uX2tkoABaR7YQ3H2bpG21j/ea2UZJi4/0wAA032G9hmBmSyWdLunB2kVXmtlj\nZnaTmc2b5rEBaLJJF4KZzZb0Q0mfc/dBSd+StEzSco3vQVxb53YrzWyNma2pKH6ODKBYkyoEMytr\nvAy+5+63SZK7D7j7qLuPSbpB0hmHuq27r3L3fnfvLys+mSqAYmULwcxM0o2SNrr7dRMuXzThahdL\nWj/9wwPQTJM5ynC2pE9JWmdmj9Yuu0rSpWa2XOOHIjdLuuKIjBBA0xxV52UAMDWshwDgsFEIABIK\nAUBCIQBIKAQACYUAIKEQACQUAoCEQgCQUAgAEgoBQEIhAEgoBAAJhQAgoRAAJE1dD8HMdkp6dsJF\nCyTtatoADh/ja0wrj6+VxyZN//iOd/eFuSs1tRBec+dma9y9v7ABZDC+xrTy+Fp5bFJx4+MpA4CE\nQgCQFF0Iqwq+/xzG15hWHl8rj00qaHyFvoYAoLUUvYcAoIVQCAASCgFAQiEASCgEAMn/A80kBGJd\nudq1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c57e6b8ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.matshow(X_train[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 style='color:purple'>Normalize training data before training the neural net</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train = X_train/255"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_test = X_test/255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 style='color:purple'>Now build the Sequential Model and add layers into it</h3>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\ProgramData\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Flatten, Dense, Activation\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Flatten(input_shape=[28, 28]))\n",
    "model.add(Dense(100, activation=\"relu\"))\n",
    "model.add(Dense(10, activation=\"softmax\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='fashion_neural_net.png' />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "flatten_1 (Flatten)          (None, 784)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 100)               78500     \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                1010      \n",
      "=================================================================\n",
      "Total params: 79,510\n",
      "Trainable params: 79,510\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(loss=\"sparse_categorical_crossentropy\", \n",
    "              optimizer=\"adam\",\n",
    "              metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\ProgramData\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\ops\\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n",
      "Epoch 1/1\n",
      "60000/60000 [==============================] - 7s 115us/step - loss: 0.5098 - acc: 0.8232\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1c50b939400>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 0s 49us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.46407912425994874, 0.8295]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Above shows accuracy score of 82.76%. The first parameter is loss**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1c50c40d748>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAECCAYAAAAYUakXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEL9JREFUeJzt3VuMXfV1x/HfmpnjmfENe/ClrrExDhdBK2HUqUlbVDki\nTV1eACmxwkPrSpWMVIpA4qGIF1ClSrQNpH1CMgXFlQgRFVBQhJoQiwaiRFbMpcbYDRDLtDbjGcz4\nMr7O5aw+zPY/E5iz/sdzLvs4fD+SNWf2Omfv5T3jn/c++3/+29xdACBJXWU3AKBzEAgAEgIBQEIg\nAEgIBAAJgQAgKSUQzGyzmf3CzD40swfL6CFiZgfN7F0ze8fMdndAP0+b2YiZ7Z2xbMDMXjWzD4qv\nSzusv0fM7HCxD98xs9tK7G+Nmb1mZvvM7D0zu69Y3hH7MOiv7fvQ2j0Owcy6Jb0v6U8kHZL0c0l3\nufu+tjYSMLODkgbd/WjZvUiSmf2xpFOS/s3df7dY9o+SRt390SJUl7r733ZQf49IOuXu3yqjp5nM\nbJWkVe7+lpktkvSmpDsk/aU6YB8G/W1Rm/dhGUcIGyV96O4H3H1c0vck3V5CH5cMd39d0uhnFt8u\naUfxeIemf4FKUaO/juHuQ+7+VvF4TNJ+SavVIfsw6K/tygiE1ZL+b8b3h1TSXz7gkn5kZm+a2bay\nm6lhpbsPFY+PSFpZZjM13Gtme4pTitJOaWYys3WSbpK0Sx24Dz/Tn9TmfcibirO7xd03SPozSfcU\nh8Qdy6fP+zptDPoTktZL2iBpSNJj5bYjmdlCSc9Lut/dT86sdcI+nKW/tu/DMgLhsKQ1M76/oljW\nMdz9cPF1RNKLmj7N6TTDxbnnhXPQkZL7+TXuPuzuU+5elfSkSt6HZlbR9D+2Z9z9hWJxx+zD2for\nYx+WEQg/l3SNmV1lZvMkfVPSyyX0MSszW1C8sSMzWyDpa5L2xq8qxcuSthaPt0p6qcRePufCP7TC\nnSpxH5qZSXpK0n53f3xGqSP2Ya3+ytiHbb/KIEnF5ZN/ltQt6Wl3//u2N1GDma3X9FGBJPVI+m7Z\n/ZnZs5I2SVomaVjSw5L+Q9JzktZK+kjSFncv5Y29Gv1t0vShrks6KOnuGefr7e7vFklvSHpXUrVY\n/JCmz9NL34dBf3epzfuwlEAA0Jl4UxFAQiAASAgEAAmBACAhEAAkpQZCBw8LlkR/jerk/jq5N6m8\n/so+QujoH4ror1Gd3F8n9yaV1F/ZgQCggzQ0MMnMNkv6F02POPxXd380ev486/U+LUjfT+i8Kuqd\n8/Zbjf4a08n9dXJvUvP7O6fTGvfzlnvenANhLhOdLLYBv9lundP2AMzdLt+pkz6aDYRGThmY6AT4\nDdNIIFwKE50AuAg9rd5AcflkmyT1aX6rNwegAY0cIdQ10Ym7b3f3QXcf7OQ3cQA0FggdPdEJgIs3\n51MGd580s7+R9AP9aqKT95rWGYC2a+g9BHd/RdIrTeoFQMkYqQggIRAAJAQCgIRAAJAQCAASAgFA\nQiAASAgEAAmBACAhEAAkBAKAhEAAkBAIABICAUBCIABICAQACYEAICEQACQEAoCEQACQEAgAEgIB\nQEIgAEgIBAAJgQAgIRAAJAQCgIRAAJAQCAASAgFAQiAASHoaebGZHZQ0JmlK0qS7DzajKQDlaCgQ\nCl9x96NNWA+AknHKACBpNBBc0o/M7E0z29aMhgCUp9FThlvc/bCZrZD0qpn9j7u/PvMJRVBsk6Q+\nzW9wcwBaqaEjBHc/XHwdkfSipI2zPGe7uw+6+2BFvY1sDkCLzTkQzGyBmS268FjS1yTtbVZjANqv\nkVOGlZJeNLML6/muu/9nU7oCUIo5B4K7H5B0YxN7AVAyLjsCSAgEAAmBACAhEAAkBAKAhEAAkDTj\n045AR7Ce+NfZp6biFbg3tP2u+fHQ/OqZM2HdbvqdsO5vv3fRPV0sjhAAJAQCgIRAAJAQCAASAgFA\nQiAASAgEAAnjEPAr03NbBPXM/x/V+Dp/9zXrw/rIppVhfcW/7wvrU8dPhPVWy40zyDmwZXFYv+rt\nhlZfF44QACQEAoCEQACQEAgAEgIBQEIgAEgIBAAJ4xBQv8w4g5wjX43HGRwbnAjrp1fF8wWs/buf\nXnRPzdRz5Zqwfvj2uF4Za2Y3c8MRAoCEQACQEAgAEgIBQEIgAEgIBAAJgQAgYRwCEuuphHWfGA/r\nE1/9vbB+4rr4vgeVT+Ltn//Subj+w3Vh/cjxRWF9fl/89zt26LKwXll6PqxftuhoWD/xcbz+dsge\nIZjZ02Y2YmZ7ZywbMLNXzeyD4uvS1rYJoB3qOWX4jqTNn1n2oKSd7n6NpJ3F9wAucdlAcPfXJY1+\nZvHtknYUj3dIuqPJfQEowVzfVFzp7kPF4yOS4kHqAC4JDV9lcHeXVPPdIjPbZma7zWz3hOI3XQCU\na66BMGxmqySp+DpS64nuvt3dB919sKLeOW4OQDvMNRBelrS1eLxV0kvNaQdAmbLjEMzsWUmbJC0z\ns0OSHpb0qKTnzOyvJH0kaUsrm0STdHWH5dw4g+4l8XXy978er98yZ4xTvfE4hf6F8QrM4td3dcX1\n3Ouvvm4orB/4eFlYP3ZiQVhXT7z9dsgGgrvfVaN0a5N7AVAyhi4DSAgEAAmBACAhEAAkBAKAhEAA\nkDAfwsUwi+ueuY6cGQcgrza0fuuJf5w+ORmvP+OXD9wQ1ntrjled1n0u3n9n1sb9ze+N79tw6JP4\nU/hd3fH+rVbj/x9Hz/THrx+Pf769i+JxFJV58d8/Nw5k6viJsF4PjhAAJAQCgIRAAJAQCAASAgFA\nQiAASAgEAMkXaxxCo+MIcvWc6lRDL2/1OIORv/7DsD6+Ih4HsGRPfF+Faua3rWdxPB/D6LF4PgE/\nNi+uXx6vv9IT/3wq3Y39/HLzMSzsj8cpTNy4Pl7/j9++6J4+t46G1wDgNwaBACAhEAAkBAKAhEAA\nkBAIABICAUDyxRqH0Og4gsx8Btadue/BZHwdP9dfo+MMhh6IxxmMXR2vv+9wPM7g/EC8fc8MA+nr\nj8cJnBpaGK9gYTxOIDfdxKmz8Z3F+nvj/pQd5pJ5QsZHm/vC+lU/bmj1kjhCADADgQAgIRAAJAQC\ngIRAAJAQCAASAgFAcmmNQ8jd1yAndyHaMvmYmc/AG5zvIKf76qvC+sFvrgrrU/2Zz+P/Mv51mIyn\nI9BUb7z+8YF4/8wbj7dvmev4Pf2ZcR4ZU1Pxz//ceDwOQ1Nxf+fPZOaLqMavv3LjoXj7TZA9QjCz\np81sxMz2zlj2iJkdNrN3ij+3tbZNAO1QzynDdyRtnmX5t919Q/Hnlea2BaAM2UBw99cljbahFwAl\na+RNxXvNbE9xShHfVA/AJWGugfCEpPWSNkgakvRYrSea2TYz221muycUTyIJoFxzCgR3H3b3KXev\nSnpS0sbgudvdfdDdByuKP00GoFxzCgQzm3l9605Je2s9F8ClIzsOwcyelbRJ0jIzOyTpYUmbzGyD\nJJd0UNLddW3N4nsLZD/v3+Lr/PLG1t+z5oqwfva6lWF99Pr4COrsb8XX+bsyH9evjMXXuccvi9c/\nuSgzX0MlM9/EvHgciGeuw192xYmw3luJf39GT8QDKaYmM/NZZPpT5r4LfjYzzqM7fv3RU3H/y//g\nxtrF//5p+NoLsoHg7nfNsviputYO4JLC0GUACYEAICEQACQEAoCEQACQEAgAkvbOh+CN3VugZ93a\nsH722hVhfWJhfJ15fEGcj5P9YVlj6+J6bj6Crom43nM6vg7umXgfXxyvf6ovrltumEh/PM7Azsb7\nf2I8/guMz4sbOD68KKxXFsdD53P3hTh9PP4FqCyIX798yamwfuJMvP7rlw2H9UMrrqlZq1bquycE\nRwgAEgIBQEIgAEgIBAAJgQAgIRAAJAQCgKSj7stw6hs3x/Xfjq9jd2Wuk59bFtc983l0y8y73zWZ\nef2p+PWTC+LXn1uZma8hd6k5Mx9B9/H41yE3zqF7YfwD6OqKtz+RuW/B2dPxfBHdJ+Pfj97lcx8D\nU4+J431hfaQa78DcOIgl886G9Y+DcSyWmariAo4QACQEAoCEQACQEAgAEgIBQEIgAEgIBABJW8ch\nVJcu0NiffrlmffIvPg1ff+qDy8N633Ccb5X44+jyrsw4g8x9D7w7MxAgU65kxilUK/Hfz+LL/JrI\n3Fch119uvgTPbN964tcPrDgZ1q+/fCTewNVxeXHlXFjvscw4jzVx+ci5xWF9RW/8Czg6Pj+sf3zm\nsrDe//HpmrWu8cwP58Lz6noWgC8EAgFAQiAASAgEAAmBACAhEAAkBAKApK3jELrHzmvJfx2oWX9/\n4/rw9Stu+CSsX/n7x+bU1wXnJuPP4w+fWRjWjx6L7wsweXxeWK9kPs9frWTGAWTGEfjARFjfsP5/\nw/ryvvg6+vr+o2F9KjOhwkPLfhHW/+HT2vcdkKQfDl8f1v/p2u+H9YHueL6FKa9zUoEazni8/39w\nJr7vyIfnVob1N5asrlnznvr+788+y8zWmNlrZrbPzN4zs/uK5QNm9qqZfVB8XVrXFgF0rHpiY1LS\nA+5+g6QvS7rHzG6Q9KCkne5+jaSdxfcALmHZQHD3IXd/q3g8Jmm/pNWSbpe0o3jaDkl3tKpJAO1x\nUW8qmtk6STdJ2iVppbsPFaUjkuITHAAdr+5AMLOFkp6XdL+7/9qnUNzdJc36jouZbTOz3Wa2e7wa\nTxIJoFx1BYKZVTQdBs+4+wvF4mEzW1XUV0ma9aNo7r7d3QfdfXBeV+b2yQBKVc9VBpP0lKT97v74\njNLLkrYWj7dKeqn57QFoJ/PMtVUzu0XSG5LelXThQ9UPafp9hOckrZX0kaQt7j4arWuxDfjNdmuj\nPdfUvTS+8nny1mvD+rFr43EAPRvjcQ5fGoivw69dEL9+dW9c7579rCyZykxoMFGNh53sO7UqrP/s\nwFVhfelr8X0Jln9vT1ivnq79ef5mqO6MJzT4yvL3w/qesdrX+SXpyOl4PoRPT8fzHUxOxr9/E+Px\nz+/ae2qP8fnZyZd0YvKT3J078gOT3P0nqj11Ruv+dQNoO4YuA0gIBAAJgQAgIRAAJAQCgIRAAJBk\nxyE0U6vHIQCY3S7fqZM+mh2HwBECgIRAAJAQCAASAgFAQiAASAgEAAmBACAhEAAkBAKAhEAAkBAI\nABICAUBCIABICAQACYEAICEQACQEAoCEQACQEAgAEgIBQEIgAEgIBAAJgQAgyQaCma0xs9fMbJ+Z\nvWdm9xXLHzGzw2b2TvHntta3C6CVeup4zqSkB9z9LTNbJOlNM3u1qH3b3b/VuvYAtFM2ENx9SNJQ\n8XjMzPZLWt3qxgC030W9h2Bm6yTdJGlXseheM9tjZk+b2dIm9wagzeoOBDNbKOl5Sfe7+0lJT0ha\nL2mDpo8gHqvxum1mttvMdk/ofBNaBtAqdQWCmVU0HQbPuPsLkuTuw+4+5e5VSU9K2jjba919u7sP\nuvtgRb3N6htAC9RzlcEkPSVpv7s/PmP5qhlPu1PS3ua3B6Cd6rnK8EeS/lzSu2b2TrHsIUl3mdkG\nSS7poKS7W9IhgLap5yrDTyTNdl/5V5rfDoAyMVIRQEIgAEgIBAAJgQAgIRAAJAQCgIRAAJAQCAAS\nAgFAQiAASAgEAAmBACAhEAAkBAKAhEAAkJi7t29jZp9I+mjGomWSjratgYtHf43p5P46uTep+f1d\n6e7Lc09qayB8buNmu919sLQGMuivMZ3cXyf3JpXXH6cMABICAUBSdiBsL3n7OfTXmE7ur5N7k0rq\nr9T3EAB0lrKPEAB0EAIBQEIgAEgIBAAJgQAg+X86ndv//oT8GAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c50c3534e0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.matshow(X_test[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "yp = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.argmax(yp[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class_labels = [\"T-shirt/top\",\"Trouser\",\"Pullover\",\"Dress\",\"Coat\",\"Sandal\",\"Shirt\",\"Sneaker\",\"Bag\",\"Ankle boot\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Ankle boot'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class_labels[np.argmax(yp[0])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 style='color:purple'>Exercise</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use keras mnist dataset from https://keras.io/datasets/ to build a neural network. Figure out,\n",
    "    1. Optiomal number of hidden layers and neurons in each hidden layer that can give you best accuracy\n",
    "    2. Optional accuracy score"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
